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Agenda 

What  are  product  lines? 

Why  a  pilot  in  measurement? 

How  did  the  pilot  apply  goal-driven  software  measurement 
to  product  line  acquisition? 

Results  and  next  steps 
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Product  Line  Framework 

A  Framework  for  Software  Product  Line  Practice 

Version  4.0 

http://www.sei.cmu.edu/plp/framework.html 

•  provides  29  practice  areas  for  software  product  lines 

Software  Product  Line  Acquisition:  A  Companion  to  A 
Framework  for  Software  Product  Line  Practice 
Version  2.0 

http://www.sei.cmu.edu/plp/companion.html 


places  practice  areas  in  acquisition  context 


—  CamegieMclkia 

— Software  Engineering  Institute 

What  are  product  lines? 

A  set  of  software-intensive  systems  sharing  a  common, 
managed  set  of  features  that  satisfy  the  specific  needs  of  a 
particular  market  segment  or  mission  and  that  are  developed 
from  a  common  set  of  core  assets  in  a  prescribed  way 


Product  Line  Definition 

For  NUWC 

Set  of  software  intensive  systems 

Data  acquisition,  display  and  control  systems 
for  ranges 

Sharing  a  common,  managed  set  of  features 

Features  for  tracking,  archiving,  playback, 
debriefing,  analysis,  other  applications  to 
support  range  operations 

(The  systems  must) 

•satisfy  the  specific  needs  of  a  selected 
market  segment  or  mission 
•(and  be)  developed  from  a  common  set  of 
core  assets 

T&E  /  training  range  community 

Rangeware  assets  including  common 
architecture,  pre-built  applications,  scripts  for 
production  builds 

In  a  prescribed  way 

NUWC  processes  for : 

•Java  development  and  maintenance  for 
RangeWare 

•Production  plan  for  range  systems 
•Configuration  management  plan 

GamegieMelloci 
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Applying  RangeWare  -  Savings 


Program 

Name 

Lines  of 
Code 
Estimated 
(in  000’s) 

Actual 

Lines 

Developed 

Estimated 
Cost  (in 
000’s) 

Actual 
Cost  (in 
000’s) 

Years  of 
development 

Subsystem  A 

250 

(Est.)  75 

1562 

ECSWTR 

245 

80 

1300 

Subsystem  B 

150 

(Est.)  45 

937 

AHRP 

165 

25 

200 

99-01 

LSVTC 

150 

20 

210 

01-02 

150 

28 

219 

01-02 

SCORE 

150 

20 

115 

01-02 

Eng  Prot  A 

150 

(Est.)  45 

937 

TSMADS 

152 

22 

540 

99-02 
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Pilot  Goals  Description 

Demonstrate  successful  product  line  acquisition  practices  for 
measurement  within  a  DoD  setting 

Goals  for  the  pilot 

•  Serve  as  example  for  future  PWS  work  by  demonstrating  the 
overall  effectiveness  of  the  Acquisition  Companion 

•  Work  hand-in-hand  with  acquisition  organization  to  obtain 
and  report  on  adoption  from  within  DoD 

•  Serve  as  example  for  other  Navy  systems  considering 
product  line  approaches 

Acquisition  pilot  allows  low  risk  opportunity  to  apply  acquisition 
practices  in  such  areas  as  Structuring  the  Organization,  Data 
Collection,  Building  a  Business  Case,  Customer  Interface 
Management,  and  Technology  Forecasting 
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Value  and  Significance 


What  is  the  value  to  the  customer? 

•  Leverages  current  improvement  plan  with  SEI  processes 

•  Makes  case  for  strategic  funding  through  measurement  and  tracking 
guidance 

•  Promotes  long-term  vision  for  sustained  product  line  development 
What  is  the  value  to  the  SEI? 

•  Leverages  development  and  validation  of  new  methods  through  work 
with  DoD  organization  and  variety  of  users 

•  Matures  Acquisition  Companion  Guidelines  through  customer  work 

•  Applies  cross-functional  teams  (PLP,  SEMA,  ASP) 

What  is  the  value  to  the  Acquisition  community? 

•  Satisfies  requests  by  others  for  support  in  same  practice  areas  (e.g., 
FBCB2) 

•  Provides  extended  case  studies  of  successful  technology  adoption 

•  Extends  the  current  NUWC  case  study  in  specific  practice  areas  for 
new  product  line  work  at  NUWC 
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Results 

Applying  Goal  Driven  Software  Measurement  to  NUWCs  product 
line  adoption 

•  NUWC  has  established  measurement  goals  and  indicator 

•  Tracking  effort  data  from  Off-board  Advanced  System 
Stimulus  (OASYS)  project 

•  Three  other  projects  will  also  be  capturing  data 

•  Measurement  group  with  representatives  from  4  projects  in 
place 

Unexpected  and  beneficial  discoveries 

•  Goal-Driven  Software  Measurement  to  provide  basis  for 
product  line  measurement  practices 

•  Project  effort  must  track  WBS,  action  items,  problems  reports, 
other  sources  of  requirements 

•  New  NUWC  projects  will  be  applying  results  of  pilot 
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Goal-Driven  Software  Measurement 


Step  1 :  Identify 
your  business 
goals 

Step  2:  Identify 
what  you  want  to 

* 

Step  3:  Identify 

your  subgoals 
(or  objectives) 

14 

Step  4:  Identify 
the  entities  and 
attributes 

1 

Formalize  your 
measurement 
goals 


Step  6:  Identify  your  measurement 
questions  &  indicators 


Step  (\  Identity  the  data 
elements 


Step  8:  Define  and  document 
measures  and  indicators 

Indicator  Template 


Goal  ID: 

Objective: 

Question: 


Inputs 

Algorithm 

Assumptions 


Step  9:  Identify  the 
actions  needed  to 
implement  the  measures 


Planning 

Data  Elements 

Tasks 

*1 

2 

3 

4 

5 

Taskl 

50 

N 

Y 

Task  2 

Y 

Y 

Y 

Task  3 

Y 

Y 

Task  n 

N 

Y 

Data 

Elements 

Size 

Defects 

Avail  Source 


+ 

-OA 

Q 

rl\i 

_ 

? 

Q 

Etc 

+ 

■  " 

Step  10:  Prepare  a  plan 


Verification  and 
action  plans 


GamegieMelloci 

Software  Engineering  Institute 


Business  Goal 

XX 

Business  Subgoals 


Become  the  best  value  supplier  of 
high  quality  range  software 


•Improve  accuracy  of  effort  estimation  process 
•Measure  effectiveness  of  Product  Line  Approach 
♦Improve  ahility  of  products  meeting  customer  requirdments 


Measurement  Goals 


Questions  &  Indicators 


Object  of  Interest  Effort  estimation  process 


Purpose 


Perspective 


Environment  and 
Constraints 


Divide  SCPs  into  units  that  are  trackable. 
Improve  ability  to  make  reliable  estimates  of 
effort. 

Analyze  causes  of  deviation  from  effort 
estimates.  Establish  correlation  between  types 
of  software  changes  to  assets  and  how  these 
affect  schedule 

Use  TrackWeb  to  estimate,  track  and  report 
effort 


How  reliable  are  schedule  estimates? 
What  causes  deviations  from  estimates? 


Data  Elements 


•  SCP  type  •  Effort  estimate  •  Degree  of  reuse 
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NUWC  Code  71  Business  Goal 


...Become  the  best  value 
supplier  of  high  quality 
range  software  systems 


Our  principal  business 
subgoals  are  ... 


Subgoal  #1:  Improve 
accuracy  of  effort 
estimation  process 

Subgoal  #2:  Measure 
effectiveness  of  Product 
Line  Approach 

Subgoal  #3:  Improve 
ability  of  products  to  meet 
customer  requirements 
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Improve  accuracy  of  effort  estimation 
process 

For  each  project: 

•  T rack  effort  as  budgeted  compared  to  effort  at 
completion 

•  Track  budgeted  schedule  and  cost  in  similar  fashion 

Use  SCP’s,  Al’s,  and  WBS  tasks  as  the  level  for  tracking 
this  information 

Across  projects,  track  variations  in  use  and  effect  of  use  of 
assets  (element  of  Subgoal  #2) 
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Other  Indicators 

Subgoal  #2 

•  Cost  savings  from  asset  use 

•  Effort  savings  from  asset  use 

Subgoal  #3 

•  %  coverage  (degree  of  reuse) 

•  Degree  of  change  of  assets  for  meeting  customer 
needs 


CarnegieMellm 

— Software  Engineering  Institute 

Necessary  Data  Elements 


Indicators 


Code 

Meaning 

+ 

Available 

0 

00 

000 

Not  explicitly  available 

-  can  be  derived  from  other  data 

-  can  be  obtained  via  minor  effort 

-  can  be  obtained  via  significant  effort 

Not  available  now 

-- 

Impossible  to  obtain  or  extremely  difficult 

Data  Elements 

Indicator 

Required 

a  b  c 

1  Identifier 

XXX 

2  Description 

XXX 

3  Requirement 

XXX 

4  Baseline  (Effort,  Completion) 

5  Revised  Baseline 

6  Number  of  Revision  (history) 

7  Rationale  for  Revision 

8  Actual  Effort  to  Date 

9  Estimate  Effort  to  Complete 

10  Est.  Actual  Completion  Date 

11  Revised  Completion  Date 

12  Labor  Resource  Cost 

13  Revised  Costs 

14  Other  Task  Cost 

15  Actuals  (effort, schedule, cost) 

X  X 

X  X 

X  X 

X  X 

X 

X 

X 

X 

X 

X 

X 

XXX 

at  complete 

Compute 

Baseline  cost  (labor  and  other) 
Costs  to  date 

%  Completed  (schedule,  effort) 

%  Actual  Effort  Spent 

Estimate  (effort,  cost)  at  complete 
Variance  (effort,  schedule,  cost)) 

X 

X 

X  X 

X  X 

X  X 

XXX 

Avail 


Source 


000 

+ 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 

00 


WBS,  SCP,  Al 
WBS,  SCP,  Al 
Requirements  doc. 
WBS  (sometimes),  Al 
Staff 


Staff  (not  collected  now) 

Staff  (not  collected  now) 

Timesheets,  staff 

Staff 

Staff 

Staff 

Staff 

Staff 

Staff 

Staff 


00 

00 

00 

00 

00 

00 


Baseline  effort  *  Labor  Cost 
Actual  Effort  *  Labor  cost 
Actual/Baseline  *100 
Actual/To  Complete  *  100 
To  Complete  -  Actual 
To  Complete/Baseline  *  100 
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Results:  Example  Indicator - 
Effort  Variance  % 

Purpose  -  measure  effort  to  completion  (estimated)  against 
baseline  effort 

EV%  =  (actual  effort  -  budgeted  effort)  /  budgeted  effort  *  1 00 
Perspective  -  analyze  when  and  why  variance  occurs 
Environment  and  Constraints  -  use  Trackweb  or  spreadsheet 
template  to  update  effort 
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Next  Steps 

Examine  other  subgoals  in  depth 

•  Develop  questions  and  indicators 

•  Identify  data  items  and  sources 

Use  measurement  results  as  basis  for  analysis  across 
projects  in  product  line 


Refine  business  case 
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Contacts 


Contact  Information 

Sholom  Cohen 

Product  Line  Systems  Program 
Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

Email:  sqc@sei.cmu.edu 
Phone:412-268-5872 

World  Wide  Web: 

http://www.sei.cmu.edu/plp 


Resources 


http://www.sei.cmu.edu/plp 


•  Detailed  software  product  line  case 
studies 

•  Software  product  line  practice 
framework  and  acquisition 
companion 

•  SEI  software  product  line  products 
and  services 

•  Info  about  courses  and  training 

•  Upcoming  events  in  the  software 
product  line  community 

•  Details  about  SPLC  2004,  Boston, 
Aug-Sep  2004. 
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